Maximum-Entropy Parameter Estimation for the k-nn Modified Value-Difference Kernel

We introduce an extension of the modified value-difference kernel of k-nn by replacing the kernel’s default class distribution matrix with the matrix produced by the maximum-entropy learning algorithm. This hybrid algorithm is tested on fifteen machine learning benchmark tasks, comparing the hybrid to standard k-nn classification and maximum-entropy-based classification. Results show that the hybrid typically outperforms the lower-scoring of the two other algorithms, often significantly; in a majority of cases the hybrid yields the highest accuracy of the three algorithms. Error analysis indicates that the hybrid’s errors overlap more with k-nn than with maximum entropy modeling.